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ABSTRACT 

This report attempts to assess the economic benefits 
and costs of the PACE (Projects to Advance Creativity in Education) 
Early Identification and Intervention Project, The report is 
structured to provide an indication of impacts generated by the 
Projects with some estimates of its direction and relative magnitude. 
The first section on impact analysis indicates a set of apparent 
relevant variables with estimates of the direction of effects (cost 
or benefits) and identifiable transfer payments. The break-even 
section indicates some benefit considerations and discusses 
break-even analysis as a way to justify project expenditures. 
Potential sources of opportunity losses are indicated. The research 
commentary section includes remarks by various participantSr which 
tend to be personal values related to social work and research 
design, However^y they have been documented to indicate major sources 
of bias and variance in the Project design, (Author/DN) 
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INTRODUCTION 



In view of the severe economic constraints, it was necessary for 
the project team to adopt a limited set of objectives. It was impossible 
to develop a full-scale benefit/cost analjicis or even a complete structure 
for analysis. 

The analysis can be viewed from several reference points; the child 
himself, actors in the surrounding environment (eg. other children, or 
other families in his neighborhood), his family, various formal governmental 
organizations (school district, city, county, state, and the office of 
education) and the general public. Often a benefit to one segment is a 
cost to another group leaving the difficult problem of estimating the social 
value of transfer pa3niients. The project could be examined internally to 
determine if the stated objectives have been met, however, this is not 
equivalent to evaluating whether the project had a net benefit to the 
general public. 

Incidental (and even accidental) side benefits may indicate a net 
positive value. However, when accounting for total social worth, the 
opportunity losses inherent in the project must be evaluated. The character 
of the opportunity losses for PACE I.D. can only be roughly identified and 
not costed precisely or accurately at this stage. 

A traditional cost/benefit analysis would have little value as the 
benefits associated with the project objectives have a time horizon beyond 
the scope of either this analysis or direct project funding. The cost 
structure of the PACE program. may be found in other documents. As no valid 
estimates of pay-offs can be made, a differential cost analysis has not been 
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performed. Lacking reliable and valid outputs and measures of effective- 
ness, tentative breakeven points are about as far as the analysis can be 
reasonably carried. 

The general conclusions of the project team are that the major 
benefits of the project lie with a social worker's support of a given 
group of families and the exposure to the potential of social work for 
personnel of the constituent school districts. A relatively small 
probabilistic return will "p^y" ^ social worker's time based on an 
estimate of economic contribution of discounted lifetime earnings. 

In this sense, the effort clearly yielded net benefits to the community 
at large. On the other hand, we do not feel that this project has provided 
an effective demonstration that the identification and intervention 
techniques utilized are more effective or economical than alternative 
methods . 

Hence, we have structured this report to provide the reader with an 
indication of impacts generated by the project, with some estimates of. 
direction and relative magnitude. The report is sectioned as: 

Impact Analysis 

Breakeven Analysis. 

Research Commentary 

Conclusions 

The first section indicates a set of apparent relevant variables 
with estimates of the direction of effects (cost or benefits) and identifi- 
able transfer payments. 

The breakeven section indicates some b?,nefit considerations and 
discusses break-even analysis as a way to justify project expenditures. 
Potential sources of opportunity losses are indicated. 



The research commentary section includes remarks by various partici- 
pants. These tend to be personal values related to social work and 
research design, however, they have been documented as they appear to 
indicate major sources of bias and variance in the project design. 



IMPACT ANALYSIS 



We have chosen to cluster the relev*-int individuals and institutions 
to avoid unnecessary redundancy of detailed transfer payments and the 
implication of precision and accuracy. 

The categories used in the Impact Matrix (see Figure 1) include: 
1) the child, 2) his family, 3) other actors (other children, neighbor- 
hood etc.), 4) Regional, (city, county, state), and 5) Federal (USOE, 
general public etc.)* 
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Figure 1 
Impact Matrix 

Relevant Parties and Organizations* 



Direct attention to 

learning /emotional problems B** B 

Reduced opportunity for 

se If -r es p ons ib i li ty *** C 
Increased learning rate B 
Less punishment for variances 

from group norms B 
Increased conditional value of 

number of years of education B B 

Reduces P (Social disability, 

mental health, delinquency, etc.) B B B . B B 

Increased lifetime standard 

of living B 

Increased positive reinforce- 
ment from society B 

Ratio of positive/reinforce- 
ment/punishment increased B 

Identified as "different" C 

Damming of aggression 

identified and dissipated B B B B 

Increased teacher time B 
Lowered distraction rate B 
Decreased variety in environment C 
Decreased emotional "costs" B 
Reduced absenteeism from school; 

(higher home costs) B T(-st£te) 

(+local) 

* 1. the child, 2. family, 3. other persons, 4. Regional government, 5. Federal 
** B = Benefit, C = Cost, T = transfer payment. 

*** Note that many if not most benefits to individuals are also social benefits to 
column 5 - only direct benefits and costs will be noted. 
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Reduced parent/teacher con- 
ferences and entire complex of 
school/parent problems 

Potential transference by 
child 

Status re neighbor reaction 
to social worker visits 

Presumed clarification of 
family problems & Proficient 

Polarization of conflict by 
introduction of new. party 

Year-round service by social 
agency 

Teacher time 

Principal time 

Service and overhead costs 

Other social workers time 

Expanded usage of school 
psychologist 

Reduced community property losses 
from juvenile delinquency (expected 
value) 

Reduction of delinquency (EV) 

Reduced loss of state aid 

Parental demands for increased 
school services 

Processing parents complaints 

Alternative uses for aid 

Reduced or shifted welfare costs 

Reduced juvenile delinquency 

Increased tax base 

All economic values accruing 
^"'•ter citizenship 
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Acceptance of social worker 
in the home 

Presumed saving 
regarding children & aid 

Demonstration effects 

Reduced R&D expense 

Test project 

Direct erst of project 

Opportunity losses 

Increased S/T ratios 

Increased teachers ' 
effectiveness 

Increased teacher mbrale 

Indicates potential resource 
shifts 

Earnings 

Intergeneration & 
effects 

Substantial ''investment'* benefits* 
Different teacher training needed 
Lower cost of hiring teachers 



J, 2 3 4 5 

B 

T T 
(B) 
(B) 

(B) 
C 

(C) (C) (C) (C) (C) 

B B 

B B B 

B 

B B 

B B B 

B B B B B 

B B B B B 

C C 

B B 



* Corrective rather than supportive therapy (see next section) 



ERIC 



BREAK EVEN ANALYSIS 



Considering the extended time horizon necessary to measure benefits, 
at best, gross estimates can be made and a break-even region approximated. 
There are a variety of direct and indirect potential measures available; 
the major categories relating to crime, health, and lifetime earnings. 
The intervening variable of educational level makes the probability of 
high school graduation as the largest incremental change. Affects of 
differential levels of education can be seen in the Spiegelman*^ report. 
Lifetime earnings 

The importance of the discount rate swamps the present value of lifvitime 
earning differentials attributed to educational level. The first break of 
relevance appears between the 10th grade and high school graduation. Using 
conservative estimates of lifetime earnings from the 1960 census (adjusted) 
and a discount rate of 2%, the difference in return. to the additional 
education has an expected value (for white males) of $9,300 discounted 
to present value • For non-white males the figure is $11,500, with a 
similar differential for females. As an indication of the sensitivity 
to the discount rate, the difference in present value reduces to approximately 
$1,000 as we raise the interest rate to 5%. 
Juvenile Crime 

Closely related to the educational level attained is the probability 
of being arrested for juvenile crime; presumably the PACE ID Program 
operates both directly and indirectly to reduce this probability. The 



^Spiegelman, Robert, et alo, "A Benefit/Cost Model to Evaluate Educa- 
tional Programs, (SRI Project 6179, prepared for the U. S. Office of 
Education), Menlo Park, California: Stanford Research Institute, 1968. 
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social returns to a reduction in juvenile crime are reduced costs to the 
victims and the costs assumed by the general populace for detection, 
detention, etc. Available estimates in California suggest a comparative 
figure of $1,390 per delinquent, thus each Pacer prevented from becoming 
a juvenile offender saves the community at least $1,000 per arrest. 
There is considerable evidence to suggest that there are repeated arrests 
for offenders and this figure may be too low. 
Probability of High School Graduation 

The effectiveness of PACE in changing the probability of high school 
education (theoretically) could be estimated by pre- and post-AML test 
score distribution adjusted by transition matricies using a Markov chain. 
Other Benefits 

The accumulated figures preclude the necessity for discussing other 
benefits such as probability of college, obvious inter-generation effects, 
or direct increases in achievement unrelated to educational attainment. 
Summary 

Therefore, making the assumption that lifetime earnings measures the 
economic value of the individual's contribution to society and using a 
discount rate of 2%, we find that if one Pacer who would have dropped out 
after the 10th grade and became a juvenile offender graduates and stays 
out of juvenile court, the social benefits, properly discounted, are some- 
where above $10,000. 

The incidental benefits of the project, if this dollar figure is 
reasonable, suggest that the value of the social worker to a community 
for (on the average) reclaiming two children a year is sufficient to 
pay for the social worker and the related overhead costs, 

ERIC 



10 



Particularly since this project is focused on potential multi-problem 
families rather than actual, it appears highly likely to this team that 
the project has paid for itself (in terms of social benefits to the 
community at large). If this is true, and if there is no loss due to 
case load and other transfer payments of this type, it is possible to 
argue that any information from the project is a windfall gain. In fact 
it costs the larger community nothing, since the project is paid for 
in terms of welfare benefits and all additional information is free. 
This is in contrast to the argument that perhaps there is an decrement 
of information lost due to opportunity costs accrued from sub-optimal 
organization within the project. 
Break-even Region 

It should be noted that the project was oriented towards both identifi- 
cation and intervention. However, the incidental accrued benefits of social 
work relate only to the intervention dimensions, that is, identification 
without intervention becomes sterile insofar as incidental benefits are 
concerned. On the other hand, there may be accrued benefits from simple 
identification pending validation of the identification instrument. For 
example, longitudinal measures taken at an appropriate time, between the 
10th and 12th grade in high school, would allow a comparison to determine 
whether the AML scale in fact identified potential high school dropouts. 
In Figure 2 we have indicated some very rough social cost estimates as 
prepared in the Spielgman report. 
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Figure 2 
ESTIMATED SOCIAL COST 

Public Health and Welfare Services (ADC) $148.00 

General Assistance (and especially alcoholics) 90.00 

Public Health Division (half-hour home visit) 12.83 

Mental Health Services (state hospital per month) 503.00 
Probation Department 

Juvenile Hall per day \7 .99 

Mean unit intake case ner 12 months 557.91 

Transfer payments lost (chronic absences) 86.60 

Educational handicapped per case 1500.00 

From these figures a very rough break-even region may be estimated, 
using as direct ead variable costs the estimates given in the final report 
of the PACE I.D. project for an ideal intervention center. This contains 
the assumption that the estimate of the existing staff relative to benefits 
accrued on returns to scale and organization system are within a region 
of satisfactory attainment (rather than optimality). 
Incommensurables 

One of the inherent problems in an analysis of this type is the 
validity of the underlying assumptions relative to the social value of 
some institutions within our 3'.flture. It is probably true that individuals 
embedded within a culture are more adapted to the degree that their values 
and those of the culture are congruent. On the other hand, it does not 
necessaril:^ follow that the culture progresses as congruency increases. 
In the evolution of a species, mutation is a necessary device for 
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adaption, and it probably should not be overlooked that a program of this 
type is basically mutation reducing. Hopefully, however, it could be 
treated as mutation selective. However, the time horizon of effective 
mutation is probably such that it will be impossible to predict and 
discriminate between desirable and undesirable mutations. 

is feasible that this is a program which will minimize teacher 
emotional costs rather than maximize benefits to future generations. 
That is, the inteir-generation effect could be actually negative. 

Perhaps a better and equally reasonable assumption to make would be 
that the human mechanism is sufficiently adaptive to survive both the 
educational system and social workers if it is necessary to do so. Indeed, 
the appropriate assumption is probably that (in the aggregate) society 
will retain the benefits of social welfare programs and ultimately discard 
the disadvantages. 



RESEARCH COMMENTARY 



The remarks in this Sssction consist of questions which have been 
raised by the large variety of persons participating in this study. 
No effort has been made to delete individual bias or to present 
counter arguments. They simply represent what appeared to be a possible 
alternative interpretations . 

Commentary has been clustered into four major headings: (1) AML 
Scale, (2) Social Worker, (3) School System, and (4) Research Design. 
These categories are somewhat arbitrary and are in reality highly 
interdependent. Nevertheless we hope this rough classification system 
will aid the reader. 
AML Scale 

The AML scale appears to be symptomatic. The assumption that an 
increment of AML could be sensitive to permanent type alterations in 
family structure and responsive to those changes seems very unlikely. 

It is implied in the system that an incremental change in the AML 
scale is roughly equivalent to an incremental cKange in long-term 
behavior toward adjustment. This assumption is highly unlikely to be 
valid and the *'within" variance on the AML scale is highly likely to 
swamp any measurable effect on long-term behavior or even lose reliability 
when used as a symptom of long- term behavior. 

It is implicitly assumed that the old way of identification was 
to use a school psychologist, and that the new way is to use the AML 
scale. The AML scale is simply a way of asking teachers which students 
need help. It would be cheaper to just ask the teachers. 
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1£ AML identified problems are sufficiently serious to warrant family 
intervention, can we reasonably expect any quick or permanent change in 
behavior of the child who is presumably a symptom anyway? That is, the 
AMI. scale purports to measure the symptoms of a serious underlying problem 
and then also purports to measure this underlying change. It seems 
highly unlikely that it can do both things. 

The AML scale does not seem adequately validated (with double blind 
tests, etc.); a more adequate testing phase seems desirable before actual 
usage. 

Note that twice as many families were marked severe by a social worker 
at the end of the project than at the starts Is this a measurement or an 
effect? 

Note that if scores are clustered by teacher rather than students, 
significance of score will go up. That is, the second-grade teacher 
at the start of the project rates three groups. A significant increase 
of the M scale for the experimental group may simply indicate an increak^,^ed 
perception by the teacher. 

The experimental group is targeted, the teacher receives feedback, 
and the teacher knows who are in the experimental group, but theoretically 
does not know who the control group is. This seems a naive assumption 
as the teachers (in effect) place the subjects into the groups. 

The AML scale measures teachers* behavior rather than child behavior, 
or at least contains in its variance a great deal of teacher behavior 
relating in particular to whether the teacher is bugged by certain types 
of behaviors. 
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AMI. may identify and communicate, but is probably questionable as 
measurer of change. There is a substantial bias problem inherent and 
there have beexi no unobtrusive measures utilized as a cross-check. 

If the objectives of the program are to reduce anti-social behavior 
in the classroom, there may be much more economical ways to do this. 
For example, centralizing problem children and policing. 

Hypothesis ; AML scores are strongly subject to teacher bias and 
teachers will literally control the scores to (perhaps knowingly) place 
children in the program. 

Social Worker 

It is assumed that social workers contacts and, therefore, costs are 
independent of one another and a stable probability distribution in the 
population. Attempts to determine a "proper" workload for social workers 
have not resulted in acceptable standards to date. 

Note that it is assumed that a $14,000 a year social worker is more 
effective than a $10,000 a year social worker. The lack of clear specific 
objectives makes this assessment of effectiveness questionable. The 
effectiveness per dollar expended is, while important, unknown. 

Note that it is assumed that a contact with a social worker reduces 
the probability of additional contacts with the social worker. This is 
highly questionable, the reverse is probably true. 

There is no estimate of marginal returns for an increased case load 
while the proposed center has a very low case load. It is assumed that 
the returns from lower case loads are more than proportionately increased. 
There is no appropriate measure for this . 
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Social workers are acting as "lawyers," and getting particular 
children attention. Note that in the experimental group 13 got into 
special classes and of the control group only 2 got into special classes* 
If we want to look at the data at^other way we can argue that social workers 
obviously increased bad behavior. Surely, social workers cin't have 
it both ways. 

If the objectives of the program are to identify families who are 
not currently, but have a relatively high disposition of becoming multi- 
problem families. There may be negative dollar benefits if all we're 
doing is getting an inevitable social problem onto welfare rolls more 
rapidly. 

The whole program is symptom suppression orieated. 

Hypothesis : (1) Receipt of services of a social worker increases 
the probability of demanding further services, (2) Amplifies existing 
problem, (3) Focuses attention of actors on the problem (may be good or 
bad). 

Hypothesis ; Attention of a social worker in the school is reinforcing 
to Che child, therefore, sustaining whatever behavior needs to be done to 
maintain attention of social worker, that is, potentially increases the 
probability of activities which will demand the attention of social workers 
by reinforcing unwanted behavior. 

Hypothesis ; Social workers have a high personal need to intervene. 

Hypothesis ; Competence (of social workers) is very critical, but 
is not well-specified. 

Hypothesis ; Major benefit from the low case-load consists of extended 
commitment, that is, year-round service, allowing the social worker to 
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be client attached, and reducing the need for the family meeting new social 
workers » also reduces the problem of data exchange between social workers. 

Hypothesis ; Small case-load allows social workers freedom to avoid 
standardization at!vd to specialize on talents. This may be better for the 
Individual family and self-actualizing for the social worker but Is not 
an economically effective use of the social workers' time. 

School System 

It Is assumed that there Is a learning rate loss In the classroom by 
other children. This may, In fact, not be true. The content of the 
material learned may be different from that planned. The amount learned 
may be greater. 

Hypothesis ; Teacher manipulates scores to focus attention on children 
that they; (1) like, (2) dislike, (3) think need help, (4) Interfere with 
teacher's sense of order, etc. 

Teachers are actually rating whether they like particular social 
workers • 

Hypothesis ; The perception of benefit from either the teacher or 
the principal Is a function of a particular social worker. 

Peripheral participants such as teachers or principals did not scale 
but dichotimized social workers Into good and bad* 

Hypothesis ; Something is wrong with the system if it needs an 
artificial mechanism to simply provide cotomunication from the teacher 
to Identify potential behavioral explosions. They might need social 
workers to cope with the explosions, but it is unusual that they would 
need an artificial device for identification. 
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Research Design 

They select the top 10% and turn social workers loose on them. There 
was no recognition of possible differential benefits at different levels 
of scores* The problem is a multiple discriminative analysis situation, 
and in fact the top 10% scores may not be the worst in the population. 

If experimental group families went to a large number of new agencies 
in the community which had not been used before, it can be argued that 
early referral would produce a beneficial trade-off as they will go to 
a medical clinic when appropriate, rather than juvpnile court. However, 
lo test was done to determine the truth of this proposition. This may 
be another incidental benefit to the program if it turns out to be educational 
in teaching the populace what welfare benefits are available. However, 
there is some question as to whether this is good or not. 

From the principals' viev^point; why does he need the whole program 
with all the other overhead cost? Why doesn't he just hire social workers. 
That is , there is no estimate of the minimum amount of social work any 
particular school has to buy to get the benefits . 

No unobtrusive measures. USOE should require an evaluation mode 
with every proposal providing measures of effectiveness before the fact . 

Program is probably "good" because it is investment oriented and 
the costs are not "sunk" in the sense that these dollars will continue 
to accrue benefits. 

The whole problem is a sort of a canonical correlation. There were 
at least some effect with multi-variante outputs as well as inputs. There 
are no measures set up to consider this problem. 
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It is impossible to develop an adequate cost-benefit study without 
priority measures set up like a "strong interference" set with automatic 
data fall-out into a model. 

No comparison with the effectiveness of social workers of this type 
versus social workers of any other type versus teacher intervention. 

Most teachers are ego involved and will introduce a problem of within 
group variance which may mask between group variance of particular AML 
ratings. 

Possible unobtrusive measures would be participation of parents 
in other school related activities, that is, what happens to the percentage 
membership attendance at, say PTA. As measures of effectiveness, there 
should be records on attendance of child, participation of parents, divorce 
rate, problems of parents, etc., usage of community facilities, (particularly 
medical clinics) , and production of tax-supported benefits rather than 
the other way around. 

No measures were taken on the negative changes as the PACE program 
has started to withdraw, that is, what is the relative investment versus 
operating effect of social workers. 

Are we training children, parents, teachers, or social workers? 

The value of the program has to turn on long-ru n behavior change. 
The identification of "bombs" is not enough; they must be defused. Any 
program that attends the individual will defuse for the short-run and 
could be done at a considerably lower dollar cost per case. 

We want to maximize the ratio of corrective to supportive therapy. 
Supportive therapy will probably be the only thing available at high 
school levels and above. 
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The value of the project turns on the identification of the probability 
of explosion, the conditional value of the explosion, and therefore, 
expected value. No measures or controls were introduced for these results. 

There are no alternative strategies of permanent tension reduction 
tested. There is no identification of tension inputs, no testing of 
dissipation of the summation of tensions in any functional way* No 
evaluation built in to estimate whether the probability of becoming a 
multi-problem family is reduced. 

Note that at an 8% discount rate the benefits of completing high school 
are actually negative. This would suggest that our system has changed 
to the point where there are negative economic returns to education, 
(obviously an extreme position) . 

Hypothesis ; The research design limited the case load and made the 
cost per case exorbitant and was itself useless. 



CONCLUSIONS 



On the balance the team feels that the general public probably got 
their money's worth out of the project (in terms of incidental benefits). 
However, the opportunity losses, in terms of additional information which 
could have been gained with the addition of unobtrusive measures planned 
before project inception, were great. 

Possible additional data would include validity and reliability 
measures of the test instrument and evaluation of potential designs to 
actually evaluate the effectiveness of the activities of the program. 
A variety of similar criticisms of this type appear relevant. 

This is not to imply that the management and operation of the actual 
PACE I.D. center was defective. Due to the lack of valid long-range 
measures we have made no analysis as to the internal effectiveness of 
the PACE I.D. project. It is interesting that none existed however. 

If these comments are viewed as an indictment they are not an indict- 
ment of PACE I.D. management, but of management control of the higher 
order. That is, the opportunity losses were substantial and It is the 
general conclusion of the project team that a higher ratio of prior planning 
to direct activities could have yielded substantially greater benefits 
differentially and on the margin from this particular project. 
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